Pandas Profiling Report for Nhabe Radar dataset

Dataset statistics

Number of variables	8
Number of observations	693500
Missing cells	2437725
Missing cells (%)	43.9%
Duplicate rows	56
Duplicate rows (%)	< 0.1%
Total size in memory	42.3 MiB
Average record size in memory	64.0 B

Variable types

Numeric	8

Alerts

Dataset has 56 (< 0.1%) duplicate rows	Duplicates
`reflectivity` is highly overall correlated with `total_power`	High correlation
`total_power` is highly overall correlated with `reflectivity`	High correlation
`reflectivity` has 586650 (84.6%) missing values	Missing
`total_power` has 552515 (79.7%) missing values	Missing
`velocity` has 645987 (93.1%) missing values	Missing
`spectrum_width` has 652573 (94.1%) missing values	Missing

Reproduction

Analysis started	2024-04-22 10:14:46.375118
Analysis finished	2024-04-22 10:14:56.719519
Duration	10.34 seconds
Software version	ydata-profiling vv4.7.0
Download configuration	config.json

longitude
Real number (ℝ)

Distinct	627735
Distinct (%)	90.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	10.697573

Minimum	7.968836
Maximum	13.349576
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	7.968836
5-th percentile	8.7681769
Q1	10.027169
median	10.682837
Q3	11.390367
95-th percentile	12.592767
Maximum	13.349576
Range	5.38074
Interquartile range (IQR)	1.3631988

Descriptive statistics

Standard deviation	1.0911675
Coefficient of variation (CV)	0.10200141
Kurtosis	-0.26207496
Mean	10.697573
Median Absolute Deviation (MAD)	0.6818275
Skewness	-0.032847474
Sum	7418766.6
Variance	1.1906465
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10.65961	1387	0.2%
10.659622	11	< 0.1%
10.659617	9	< 0.1%
10.659519	8	< 0.1%
10.659632	7	< 0.1%
10.658484	7	< 0.1%
10.659332	7	< 0.1%
10.659641	7	< 0.1%
10.65975	7	< 0.1%
10.659512	6	< 0.1%
Other values (627725)	692044	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
7.968836	1	< 0.1%
7.969162	1	< 0.1%
7.969193	1	< 0.1%
7.970259	1	< 0.1%
7.9705744	1	< 0.1%
7.9715533	1	< 0.1%
7.9717216	1	< 0.1%
7.97197	1	< 0.1%
7.9725876	1	< 0.1%
7.972898	1	< 0.1%

Value	Count	Frequency (%)
13.349576	1	< 0.1%
13.349453	1	< 0.1%
13.349276	1	< 0.1%
13.348269	1	< 0.1%
13.34808	1	< 0.1%
13.347501	1	< 0.1%
13.347272	1	< 0.1%
13.346938	1	< 0.1%
13.346316	1	< 0.1%
13.346261	1	< 0.1%

latitude
Real number (ℝ)

Distinct	387885
Distinct (%)	55.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	106.7564

Minimum	103.99045
Maximum	109.46606
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	103.99045
5-th percentile	104.771
Q1	106.05762
median	106.75066
Q3	107.47169
95-th percentile	108.69756
Maximum	109.46606
Range	5.475614
Interquartile range (IQR)	1.4140775

Descriptive statistics

Standard deviation	1.1224184
Coefficient of variation (CV)	0.010513828
Kurtosis	-0.30005601
Mean	106.7564
Median Absolute Deviation (MAD)	0.707512
Skewness	-0.041862169
Sum	74035562
Variance	1.259823
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
106.728325	1387	0.2%
106.73381	22	< 0.1%
106.72947	21	< 0.1%
106.7268	20	< 0.1%
106.73201	18	< 0.1%
106.732925	18	< 0.1%
106.72451	18	< 0.1%
106.73116	17	< 0.1%
106.72284	17	< 0.1%
106.73006	16	< 0.1%
Other values (387875)	691946	99.8%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
103.99045	1	< 0.1%
103.990486	1	< 0.1%
103.991234	1	< 0.1%
103.991394	1	< 0.1%
103.99291	1	< 0.1%
103.99304	1	< 0.1%
103.99327	1	< 0.1%
103.99329	1	< 0.1%
103.99404	1	< 0.1%
103.9942	1	< 0.1%

Value	Count	Frequency (%)
109.466064	1	< 0.1%
109.466	1	< 0.1%
109.46522	1	< 0.1%
109.46513	1	< 0.1%
109.46354	1	< 0.1%
109.46345	1	< 0.1%
109.46344	1	< 0.1%
109.46342	1	< 0.1%
109.4627	1	< 0.1%
109.46254	1	< 0.1%

altitude
Real number (ℝ)

Distinct	23163
Distinct (%)	3.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	7134.5652

Minimum	10
Maximum	25055
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	10
5-th percentile	368
Q1	2396
median	5784
Q3	10797
95-th percentile	18434
Maximum	25055
Range	25045
Interquartile range (IQR)	8401

Descriptive statistics

Standard deviation	5732.4764
Coefficient of variation (CV)	0.80347944
Kurtosis	-0.039298383
Mean	7134.5652
Median Absolute Deviation (MAD)	3898
Skewness	0.85495197
Sum	4.9478209 × 10⁹
Variance	32861286
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10	1387	0.2%
42	299	< 0.1%
35	289	< 0.1%
26	252	< 0.1%
85	246	< 0.1%
47	237	< 0.1%
58	234	< 0.1%
25	199	< 0.1%
289	196	< 0.1%
60	194	< 0.1%
Other values (23153)	689967	99.5%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
10	1387	0.2%
13	39	< 0.1%
14	136	< 0.1%
15	101	< 0.1%
16	23	< 0.1%
17	34	< 0.1%
18	71	< 0.1%
19	61	< 0.1%
20	72	< 0.1%
21	66	< 0.1%

Value	Count	Frequency (%)
25055	2	< 0.1%
24998	1	< 0.1%
24994	2	< 0.1%
24941	1	< 0.1%
24937	1	< 0.1%
24933	2	< 0.1%
24898	1	< 0.1%
24880	1	< 0.1%
24877	1	< 0.1%
24873	2	< 0.1%

reflectivity
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	5561
Distinct (%)	5.2%
Missing	586650
Missing (%)	84.6%
Infinite	0
Infinite (%)	0.0%
Mean	3.9302095

Minimum	-23.69
Maximum	53.25
Zeros	30
Zeros (%)	< 0.1%
Negative	40046
Negative (%)	5.8%
Memory size	5.3 MiB

Quantile statistics

Minimum	-23.69
5-th percentile	-14.26
Q1	-5.09
median	4.2
Q3	11.82
95-th percentile	23.6
Maximum	53.25
Range	76.94
Interquartile range (IQR)	16.91

Descriptive statistics

Standard deviation	11.438532
Coefficient of variation (CV)	2.9104128
Kurtosis	-0.60014493
Mean	3.9302095
Median Absolute Deviation (MAD)	8.41
Skewness	0.13062158
Sum	419942.89
Variance	130.84002
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
2.91	56	< 0.1%
8.6	55	< 0.1%
8.69	54	< 0.1%
5.55	53	< 0.1%
5.88	53	< 0.1%
6.26	53	< 0.1%
6.97	53	< 0.1%
5.49	51	< 0.1%
5.06	50	< 0.1%
7.23	50	< 0.1%
Other values (5551)	106322	15.3%
(Missing)	586650	84.6%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-23.69	1	< 0.1%
-23.61	1	< 0.1%
-23.34	1	< 0.1%
-23.25	1	< 0.1%
-23.13	1	< 0.1%
-23.06	1	< 0.1%
-22.94	1	< 0.1%
-22.76	1	< 0.1%
-22.74	1	< 0.1%
-22.68	1	< 0.1%

Value	Count	Frequency (%)
53.25	1	< 0.1%
51.4	1	< 0.1%
44.35	1	< 0.1%
43.43	1	< 0.1%
42.26	1	< 0.1%
42.1	1	< 0.1%
41.96	1	< 0.1%
41.47	1	< 0.1%
41.18	1	< 0.1%
40.79	1	< 0.1%

total_power
Real number (ℝ)

HIGH CORRELATION MISSING

Distinct	7174
Distinct (%)	5.1%
Missing	552515
Missing (%)	79.7%
Infinite	0
Infinite (%)	0.0%
Mean	8.5779023

Minimum	-18.81
Maximum	74.4
Zeros	43
Zeros (%)	< 0.1%
Negative	38297
Negative (%)	5.5%
Memory size	5.3 MiB

Quantile statistics

Minimum	-18.81
5-th percentile	-9.728
Q1	-0.81
median	7.31
Q3	16.43
95-th percentile	30.64
Maximum	74.4
Range	93.21
Interquartile range (IQR)	17.24

Descriptive statistics

Standard deviation	12.747113
Coefficient of variation (CV)	1.4860408
Kurtosis	0.63771601
Mean	8.5779023
Median Absolute Deviation (MAD)	8.57
Skewness	0.6648696
Sum	1209355.6
Variance	162.48888
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
23.36	68	< 0.1%
7.41	67	< 0.1%
23.32	66	< 0.1%
3.54	65	< 0.1%
6.79	64	< 0.1%
23.33	64	< 0.1%
23.34	63	< 0.1%
23.35	62	< 0.1%
9.91	62	< 0.1%
7.99	62	< 0.1%
Other values (7164)	140342	20.2%
(Missing)	552515	79.7%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-18.81	1	< 0.1%
-18.77	1	< 0.1%
-18.64	1	< 0.1%
-18.62	1	< 0.1%
-18.38	1	< 0.1%
-18.36	1	< 0.1%
-18.31	1	< 0.1%
-18.22	1	< 0.1%
-18.2	2	< 0.1%
-18.17	1	< 0.1%

Value	Count	Frequency (%)
74.4	1	< 0.1%
73.74	1	< 0.1%
72.78	1	< 0.1%
71.12	1	< 0.1%
70.64	1	< 0.1%
70.36	1	< 0.1%
69.54	1	< 0.1%
69.37	1	< 0.1%
69.16	1	< 0.1%
68.93	1	< 0.1%

velocity
Real number (ℝ)

MISSING

Distinct	805
Distinct (%)	1.7%
Missing	645987
Missing (%)	93.1%
Infinite	0
Infinite (%)	0.0%
Mean	-0.11849115

Minimum	-4.02
Maximum	4.02
Zeros	41
Zeros (%)	< 0.1%
Negative	24761
Negative (%)	3.6%
Memory size	5.3 MiB

Quantile statistics

Minimum	-4.02
5-th percentile	-3.68
Q1	-2.31
median	-0.24
Q3	2.07
95-th percentile	3.64
Maximum	4.02
Range	8.04
Interquartile range (IQR)	4.38

Descriptive statistics

Standard deviation	2.4225201
Coefficient of variation (CV)	-20.444734
Kurtosis	-1.2984033
Mean	-0.11849115
Median Absolute Deviation (MAD)	2.18
Skewness	0.069209687
Sum	-5629.87
Variance	5.8686035
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
-3.64	91	< 0.1%
3.25	91	< 0.1%
-3.62	91	< 0.1%
-3.68	90	< 0.1%
-2.86	90	< 0.1%
-3.42	90	< 0.1%
2.83	89	< 0.1%
-2.92	87	< 0.1%
3.07	87	< 0.1%
-2.91	87	< 0.1%
Other values (795)	46620	6.7%
(Missing)	645987	93.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
-4.02	30	< 0.1%
-4.01	65	< 0.1%
-4	56	< 0.1%
-3.99	56	< 0.1%
-3.98	64	< 0.1%
-3.97	72	< 0.1%
-3.96	73	< 0.1%
-3.95	69	< 0.1%
-3.94	60	< 0.1%
-3.93	70	< 0.1%

Value	Count	Frequency (%)
4.02	25	< 0.1%
4.01	68	< 0.1%
4	72	< 0.1%
3.99	62	< 0.1%
3.98	65	< 0.1%
3.97	71	< 0.1%
3.96	62	< 0.1%
3.95	61	< 0.1%
3.94	68	< 0.1%
3.93	73	< 0.1%

spectrum_width
Real number (ℝ)

MISSING

Distinct	214
Distinct (%)	0.5%
Missing	652573
Missing (%)	94.1%
Infinite	0
Infinite (%)	0.0%
Mean	0.73104796

Minimum	0.01
Maximum	2.45
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	0.01
5-th percentile	0.28
Q1	0.53
median	0.71
Q3	0.91
95-th percentile	1.26
Maximum	2.45
Range	2.44
Interquartile range (IQR)	0.38

Descriptive statistics

Standard deviation	0.30250983
Coefficient of variation (CV)	0.41380298
Kurtosis	0.75657234
Mean	0.73104796
Median Absolute Deviation (MAD)	0.19
Skewness	0.45237358
Sum	29919.6
Variance	0.091512196
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.6	632	0.1%
0.61	615	0.1%
0.65	613	0.1%
0.76	606	0.1%
0.69	591	0.1%
0.71	584	0.1%
0.75	573	0.1%
0.59	572	0.1%
0.67	569	0.1%
0.7	559	0.1%
Other values (204)	35013	5.0%
(Missing)	652573	94.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
0.01	541	0.1%
0.02	2	< 0.1%
0.03	2	< 0.1%
0.04	8	< 0.1%
0.05	8	< 0.1%
0.06	6	< 0.1%
0.07	21	< 0.1%
0.08	18	< 0.1%
0.09	38	< 0.1%
0.1	29	< 0.1%

Value	Count	Frequency (%)
2.45	1	< 0.1%
2.42	1	< 0.1%
2.27	2	< 0.1%
2.22	3	< 0.1%
2.19	1	< 0.1%
2.14	4	< 0.1%
2.12	2	< 0.1%
2.11	1	< 0.1%
2.09	2	< 0.1%
2.08	1	< 0.1%

time
Real number (ℝ)

Distinct	156
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	88.093006

Minimum	1
Maximum	171
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	5.3 MiB

Quantile statistics

Minimum	1
5-th percentile	8
Q1	49
median	90
Q3	133
95-th percentile	164
Maximum	171
Range	170
Interquartile range (IQR)	84

Descriptive statistics

Standard deviation	49.742886
Coefficient of variation (CV)	0.56466328
Kurtosis	-1.1675292
Mean	88.093006
Median Absolute Deviation (MAD)	42
Skewness	-0.066255634
Sum	61092500
Variance	2474.3547
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
62	5500	0.8%
96	5500	0.8%
82	5500	0.8%
79	5500	0.8%
169	5500	0.8%
104	5500	0.8%
50	5500	0.8%
156	5500	0.8%
5	5500	0.8%
52	5000	0.7%
Other values (146)	639000	92.1%

Minimum 10 values
Maximum 10 values

Value	Count	Frequency (%)
1	4000	0.6%
2	5000	0.7%
3	4000	0.6%
4	4500	0.6%
5	5500	0.8%
6	4000	0.6%
7	4500	0.6%
8	4000	0.6%
9	4500	0.6%
10	5000	0.7%

Value	Count	Frequency (%)
171	4000	0.6%
170	4000	0.6%
169	5500	0.8%
168	4500	0.6%
167	4000	0.6%
166	4500	0.6%
165	4000	0.6%
164	5000	0.7%
163	5000	0.7%
162	4000	0.6%

Auto

Heatmap
Table

	altitude	latitude	longitude	reflectivity	spectrum_width	time	total_power	velocity
altitude	1.000	0.005	-0.008	0.365	0.041	0.456	-0.159	-0.030
latitude	0.005	1.000	-0.036	-0.140	-0.010	-0.152	-0.068	0.064
longitude	-0.008	-0.036	1.000	0.181	0.004	0.030	0.050	-0.199
reflectivity	0.365	-0.140	0.181	1.000	0.129	-0.181	0.783	-0.022
spectrum_width	0.041	-0.010	0.004	0.129	1.000	-0.079	0.130	0.042
time	0.456	-0.152	0.030	-0.181	-0.079	1.000	-0.154	-0.007
total_power	-0.159	-0.068	0.050	0.783	0.130	-0.154	1.000	-0.001
velocity	-0.030	0.064	-0.199	-0.022	0.042	-0.007	-0.001	1.000

Count
Matrix

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

First rows
Last rows

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
0	10.659610	106.728325	10.0	4.67	23.08	NaN	NaN	3.0
1	10.665005	106.728330	17.0	29.22	43.99	NaN	NaN	3.0
2	10.670401	106.728340	25.0	24.00	36.10	NaN	NaN	3.0
3	10.675796	106.728340	33.0	10.53	26.90	NaN	NaN	3.0
4	10.681191	106.728350	42.0	NaN	25.88	NaN	NaN	3.0
5	10.686586	106.728355	50.0	NaN	22.91	NaN	NaN	3.0
6	10.691982	106.728360	58.0	NaN	19.56	NaN	NaN	3.0
7	10.697377	106.728360	66.0	NaN	57.62	NaN	NaN	3.0
8	10.702773	106.728360	75.0	NaN	42.19	NaN	NaN	3.0
9	10.708169	106.728370	83.0	NaN	55.11	NaN	NaN	3.0

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time
693490	13.291719	106.687490	23080.0	NaN	NaN	NaN	NaN	162.0
693491	13.297075	106.687416	23137.0	NaN	NaN	NaN	NaN	162.0
693492	13.302430	106.687330	23195.0	NaN	NaN	NaN	NaN	162.0
693493	13.307786	106.687240	23252.0	NaN	NaN	NaN	NaN	162.0
693494	13.313141	106.687160	23310.0	NaN	NaN	NaN	NaN	162.0
693495	13.318499	106.687080	23367.0	NaN	NaN	NaN	NaN	162.0
693496	13.323854	106.687000	23425.0	NaN	NaN	NaN	NaN	162.0
693497	13.329210	106.686905	23482.0	NaN	NaN	NaN	NaN	162.0
693498	13.334565	106.686830	23541.0	NaN	NaN	NaN	NaN	162.0
693499	13.339920	106.686745	23598.0	NaN	NaN	NaN	NaN	162.0

Most frequently occurring

	longitude	latitude	altitude	reflectivity	total_power	velocity	spectrum_width	time	# duplicates
5	10.65961	106.728325	10.0	NaN	22.88	NaN	NaN	37.0	3
6	10.65961	106.728325	10.0	NaN	22.93	NaN	NaN	39.0	3
29	10.65961	106.728325	10.0	NaN	23.28	NaN	NaN	74.0	3
0	10.65961	106.728325	10.0	NaN	17.20	NaN	NaN	145.0	2
1	10.65961	106.728325	10.0	NaN	17.24	NaN	NaN	146.0	2
2	10.65961	106.728325	10.0	NaN	17.25	NaN	NaN	146.0	2
3	10.65961	106.728325	10.0	NaN	22.86	NaN	NaN	36.0	2
4	10.65961	106.728325	10.0	NaN	22.87	NaN	NaN	36.0	2
7	10.65961	106.728325	10.0	NaN	22.99	NaN	NaN	23.0	2
8	10.65961	106.728325	10.0	NaN	23.03	NaN	NaN	15.0	2

Overview

Variables

Interactions

Correlations

Missing values

Sample

Duplicate rows

Most frequently occurring